Skip to content

Conversation

@jsamuel1
Copy link
Contributor

Overview

This RFC proposes enhancing the fs_read and fs_write tools to support batch operations on multiple files in a single call, with the ability to perform multiple edits per file, maintain line number integrity through proper edit ordering, and perform search/replace operations across files in a folder using wildcard patterns with sed-like syntax.

Key Features

  1. Batch Reading: Read multiple files in a single operation
  2. Multiple Edits Per File: Make multiple changes to a file while maintaining line number integrity
  3. Pattern-Based Search and Replace: Perform sed-like operations across multiple files matching a pattern
  4. File Versioning: Track file versions with content hashes and last modified timestamps
  5. Safety Features: Content hash verification and dry run mode

Implementation Staging

The implementation is broken down into three phases:

  1. Phase 1: fs_read batch operations
  2. Phase 2: Pattern replacement for fs_write using the sd crate
  3. Phase 3: Multi-file operations for fs_write

Benefits

  • Reduced verbosity in Amazon Q responses
  • Improved performance by reducing tool invocation overhead
  • Simplified error handling across related operations
  • Better atomicity for related file operations
  • Enhanced conversation history management with file versioning

Future Possibilities

The RFC also outlines future possibilities including CRDTs for versioned multi-agent changes, transaction support, conditional edits, and more advanced features.

🤖 Assisted by Amazon Q Developer

@codecov-commenter
Copy link

codecov-commenter commented May 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 16.75%. Comparing base (0a4530a) to head (103c8d5).
Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1772   +/-   ##
=======================================
  Coverage   16.75%   16.75%           
=======================================
  Files         213      213           
  Lines       20704    20704           
  Branches      871      871           
=======================================
  Hits         3468     3468           
  Misses      17236    17236           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jsamuel1
Copy link
Contributor Author

Created issue #1773 to track the implementation of this feature once the RFC is approved.

@jsamuel1 jsamuel1 force-pushed the feature/batch-file-operations branch 2 times, most recently from cc7f635 to 0d8c4ee Compare May 15, 2025 04:01
kensave and others added 4 commits May 16, 2025 18:26
* [feat]: Add a new semantic_search_client crate that provides vector embedding and
    semantic search capabilities for the Amazon Q CLI. This implementation:

    - Supports text embedding generation using Candle and ONNX runtimes
    - Provides hardware acceleration via Metal on macOS
    - Implements efficient vector indexing for semantic search
    - Includes file processing utilities for various file types
    - Supports persistent storage of semantic contexts
    - Includes comprehensive test coverage

    This crate will enable memory bank functionality for Amazon Q, allowing
    users to create, manage, and search through semantic memory contexts.

    🤖 Assisted by [Amazon Q Developer](https://aws.amazon.com/q/developer)

* Update semantic_search_client dependencies in Cargo.toml

* Refactor embedder implementation for Linux platforms to use trait objects

This change modifies the semantic search client to use Box<dyn TextEmbedderTrait>
on Linux platforms instead of directly using CandleTextEmbedder. This provides
more flexibility and consistency with the implementation on macOS and Windows,
allowing for better extensibility and polymorphic behavior across all platforms.

* Update Cargo.lock file

* Remove redundant CandleTextEmbedder import for non-macOS/Windows platforms

* fix(semantic_search): Update conditional compilation flags for embedders

Update conditional compilation flags to match the new embedding model selection logic:
- Replace target_env="musl" conditions with target_os conditions
- Update TextEmbedder trait implementation to use macOS/Windows condition
- Ensure consistent conditions across all files

🤖 Assisted by [Amazon Q Developer](https://aws.amazon.com/q/developer)

---------

Co-authored-by: Kenneth Sanchez V <[email protected]>
Co-authored-by: Kenneth Sanchez V <[email protected]>
* fix Build

* fix: Removes flakey test

---------

Co-authored-by: Kenneth Sanchez V <[email protected]>
@jsamuel1 jsamuel1 force-pushed the feature/batch-file-operations branch from 0d8c4ee to 3c14966 Compare May 17, 2025 08:30
Implement comprehensive batch operations for fs_read and fs_write tools:

- Add operations-based structure to fs_read with multiple operation types
- Implement batch processing for fs_write to handle multiple files and edits
- Add BatchReadResult and BatchWriteResult for consistent response formats
- Standardize tool summary display with improved parameter handling
- Add comprehensive test coverage for all new functionality
- Optimize performance by reducing separate tool calls for multiple files
- Improve error handling and reporting for batch operations

This feature significantly improves performance and usability when working with
multiple files by reducing the number of separate tool calls needed.

🤖 Assisted by [Amazon Q Developer](https://aws.amazon.com/q/developer)
@jsamuel1 jsamuel1 force-pushed the feature/batch-file-operations branch from c5f1a6a to e910513 Compare May 18, 2025 07:52
Implement improved edit ordering approach for fs_write tool:
- Sort operations by type priority (Create/Rewrite first, line-based operations, string operations, append last)
- Give ReplaceLines, DeleteLines, and Insert operations the same priority
- Apply line-based operations from bottom to top to avoid line number shifting
- Eliminate redundant line number adjustment logic

Enhance UI formatting:
- Use green ticks for successful operations
- Use red exclamation marks for errors
- Use neutral bullets for summary lines
- Only color the symbols, not the text that follows them

Update tool_index.json documentation to clearly explain the edit ordering approach
and explicitly state that LLMs should not attempt to adjust line numbers.

Document the approach in code-roast.md for future reference.

🤖 Assisted by [Amazon Q Developer](https://aws.amazon.com/q/developer)
@jsamuel1 jsamuel1 enabled auto-merge (squash) May 19, 2025 07:01
@jsamuel1 jsamuel1 changed the title RFC: Batch File Operations Batch File Operations for fs_read and fs_write May 19, 2025
Removes an unused backup file (fs_write.rs.bak) and updates the tool descriptions in tool_index.json to improve guidance for fs_read and fs_write tools. Also updates the batch file operations RFC to reflect current implementation status.

- Removed: Deleted obsolete fs_write.rs.bak file
- Updated: Enhanced tool descriptions with better batching guidance
- Updated: RFC documentation to accurately reflect implementation status
- Added: Implementation status section to RFC with next steps

🤖 Assisted by [Amazon Q Developer](https://aws.amazon.com/q/developer)
@dingfeli
Copy link
Contributor

This feature has been implemented.

@dingfeli dingfeli closed this Oct 17, 2025
auto-merge was automatically disabled October 17, 2025 21:42

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants